Skip to content

Fix invalid Codex Standard/Fast cost splits - #2774

Closed
Yuxin-Qiao wants to merge 1 commit into
steipete:mainfrom
Yuxin-Qiao:codex/fix-2754-split-overcount
Closed

Fix invalid Codex Standard/Fast cost splits#2774
Yuxin-Qiao wants to merge 1 commit into
steipete:mainfrom
Yuxin-Qiao:codex/fix-2754-split-overcount

Conversation

@Yuxin-Qiao

Copy link
Copy Markdown
Contributor

Summary

When the cached Standard/Fast split maps are inflated by copied fork/subagent rows, the report currently prefers standardCost + priorityCost over the canonical day/model total. This narrow report-boundary fix treats that split as untrusted and falls back to the existing canonical base-cost or canonical-token pricing path.

  • In buildCodexReportFromCache, a split is no longer published when the known standardTokens + priorityTokens exceeds the canonical totalTokens for the same day/model, or when the Int token sum overflows.
  • Untrusted splits publish nil Standard/Fast token and cost fields and reuse the existing cachedBaseCost / canonical-token pricing fallback. No proportional scaling, no ownership guessing.
  • Valid splits that reconcile with canonical ownership keep their exact token and cost values (covered by regression tests).

Tests

  • swift test --filter CostUsageScannerPriorityTests: 20 passed (3 new: invalid split, valid split, Int overflow)
  • make check: SwiftFormat clean, strict SwiftLint clean, parser hash regenerated (covers CostUsageScanner+CacheHelpers.swift)
  • make test: 69/69 shards passed, 822 selections, 0 failures
  • git diff --check: clean

Closes #2754

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: deacc86536

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1700 to +1702
let splitIsUntrusted = splitTokenOverflow || splitTokenTotal > totalTokens
let standardCost = splitIsUntrusted ? nil : cachedStandardCost
let priorityCost = splitIsUntrusted ? nil : cachedPriorityCost

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Discard the surcharge when rejecting an inflated split

When copied priority rows make the split exceed the canonical token total, this marks the split costs as nil, but that newly activates the fallback at lines 1719–1723, which adds codexPrioritySurchargeNanosByDayModel to the base cost. That surcharge is generated from the same per-file priority rows as codexModeSplitMaps, so those copied rows can inflate it too; affected fork/subagent caches therefore continue to overreport cost even though their Standard/Fast fields disappear. The untrusted path should also discard the cached surcharge or recompute it from canonical ownership.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Aug 8, 2026
@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 8, 2026, 7:41 PM ET / 23:41 UTC.

ClawSweeper review

What this changes

This PR hides Standard/Fast cost and token splits whose cached token sum exceeds the canonical daily model total, then falls back to aggregate pricing.

Merge readiness

Blocked until real behavior proof from a real setup is added - 7 items remain

Keep open, but block merge: the prior P1 remains on the unchanged head, and the branch targets a cost-cache flow that current main has since replaced. Likely related people: steipete (high-confidence current-path owner) and Xu Xiang (medium-confidence adjacent fork-accounting contributor).

Priority: P2
Reviewed head: deacc8653606437b626d9f3efc49616523cbb53f

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The patch has focused tests, but an unresolved P1 correctness defect and no real after-fix behavior proof make it not ready to merge.
Proof confidence 🧂 unranked krab (1/6) Needs real behavior proof before merge: The PR body reports tests only; add redacted after-fix CLI or runtime output from an inflated real cache showing canonical fallback, without private paths, identifiers, or account data. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🧂 unranked krab (1/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The PR body reports tests only; add redacted after-fix CLI or runtime output from an inflated real cache showing canonical fallback, without private paths, identifiers, or account data. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 4 items Unresolved surcharge path: On the PR head, an untrusted split clears its split costs, which activates the fallback that adds the cached priority surcharge; that surcharge is derived from the same copied physical rows and can remain inflated.
Current-main architecture changed: Current main now constructs the report from read-time row pricing, so this branch's cache-split helper path has been replaced and GitHub reports the branch as dirty against main.
Feature-history provenance: The current read-time pricing path dates to the cost-pricing-race repair, which removed the old surcharge and cached split-cost helpers; steipete is the strongest routing candidate for the rebase direction.
Findings 1 actionable finding [P1] Reject the cached Fast surcharge with an untrusted split
Security None None.

How this fits together

CodexBar converts locally cached Codex usage into daily cost reports consumed by the menu-bar app and CLI. Canonical day/model totals and optional Standard/Fast row pricing must agree so copied fork or subagent rows cannot inflate displayed estimates.

flowchart LR
A[Cached Codex usage] --> B[Canonical day model totals]
A --> C[Standard Fast row data]
B --> D[Cost report builder]
C --> E[Ownership validation]
E --> D
D --> F[Daily cost estimate]
F --> G[Menu bar and CLI views]
Loading

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The PR body reports tests only; add redacted after-fix CLI or runtime output from an inflated real cache showing canonical fallback, without private paths, identifiers, or account data. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Reject the cached Fast surcharge with an untrusted split (P1) - The new guard clears the split at this point, but then activates the fallback at lines 1719–1723, which adds prioritySurchargeNanosByDayModel. That value is derived from the same copied priority rows, so the affected cache can still overreport cost. This is the unresolved P1 from the previous review cycle.
  • Resolve merge risk (P2) - An inflated split can still overreport the aggregate cost because the fallback re-adds a priority surcharge derived from the same copied rows.
  • Resolve merge risk (P1) - The branch is dirty against main and its target helper flow was removed by the current read-time pricing refactor, so it cannot be safely merged without rebasing the repair.
  • Complete next step (P2) - Do not dispatch a competing repair: this branch needs contributor proof and a rebase, while the open replacement at Fix inflated Codex Standard/Fast cost splits #2782 already owns the same correction.
  • Improve patch quality - Rebase onto current main and prevent all copied-row pricing, including the Fast surcharge, from surviving an invalid-ownership fallback.
  • Improve patch quality - Add redacted real cache or CLI evidence showing an inflated split now produces the canonical aggregate estimate.

Findings

  • [P1] Reject the cached Fast surcharge with an untrusted split — Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+CacheHelpers.swift:1700-1702
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production versus test delta production +12/-5, tests +127 The focused regression coverage is substantial, but it does not exercise the nonzero surcharge that remains reachable after the new guard.

Merge-risk options

Maintainer options:

  1. Rebase and reject all untrusted row pricing (recommended)
    Port the guard onto current main and ensure an invalid split cannot contribute either Standard/Fast costs or a cached Fast surcharge.
  2. Pause in favor of the replacement repair
    Keep this branch unmerged while maintainers evaluate the open replacement at Fix inflated Codex Standard/Fast cost splits #2782, which addresses the same full-price fallback.

Technical review

Best possible solution:

Rebase the fix onto the current read-time pricing architecture and reject the entire row-derived price, including any Fast uplift, when row tokens exceed canonical ownership; preserve aggregate canonical pricing as the fallback.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: a cache with an inflated split and nonzero cached priority surcharge follows the PR’s fallback into another copied-row-derived cost. The added fixture omits that surcharge, so it does not establish after-fix recovery.

Is this the best way to solve the issue?

No: hiding the split fields alone is insufficient because the fallback retains a related untrusted surcharge. The whole row-derived price must be rejected or recomputed from canonical ownership after rebasing.

Full review comments:

  • [P1] Reject the cached Fast surcharge with an untrusted split — Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+CacheHelpers.swift:1700-1702
    The new guard clears the split at this point, but then activates the fallback at lines 1719–1723, which adds prioritySurchargeNanosByDayModel. That value is derived from the same copied priority rows, so the affected cache can still overreport cost. This is the unresolved P1 from the previous review cycle.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 617c292f650a.

Labels

Label justifications:

  • P2: This is a bounded but user-visible accuracy defect in Codex spend estimates.
  • merge-risk: 🚨 other: Merging the current patch can continue to publish inflated cost estimates for copied fork or subagent rows.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports tests only; add redacted after-fix CLI or runtime output from an inflated real cache showing canonical fallback, without private paths, identifiers, or account data. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

What I checked:

Likely related people:

  • steipete: Authored the current-main read-time pricing refactor that replaced the branch’s cached cost-split flow. (role: current pricing-path author; confidence: high; commits: 0aafe2b80563, 26fd0bbd7eed; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+CacheHelpers.swift)
  • Xu Xiang: Recent history on the same cost helper includes fork catch-up and fork-accuracy repairs. (role: adjacent fork-accounting contributor; confidence: medium; commits: 4f99e6aba8c3, 2920019bc16d; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+CacheHelpers.swift)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (2 earlier review cycles)
  • reviewed 2026-08-08T17:43:47.289Z sha deacc86 :: needs real behavior proof before merge. :: [P1] Exclude the surcharge when rejecting an inflated split
  • reviewed 2026-08-08T18:22:26.142Z sha deacc86 :: needs real behavior proof before merge. :: [P1] Exclude the surcharge when rejecting an inflated split

Copy link
Copy Markdown
Contributor Author

Closing in favor of #2782, which ports the fix to the current read-time pricing path and addresses the copied-row Fast surcharge as well. Thanks for carrying forward and crediting the initial fix/regression-test approach from this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression in v0.48.0: Standard/Fast cost split re-inflates deduplicated fork usage

1 participant